Positioning method for searching a previously registered standard pattern out of an inspection target image

ABSTRACT

A feature portion desired by a user out of an inspection target image is accurately positioned. A standard region is set so as to surround a standard pattern in a standard image of a product to be a standard for an inspection target. A sorting region, which is a region for sorting a plurality of candidates similar to the standard pattern, is set in the inspection image. The standard pattern is searched from the inspection image, to extract a plurality of candidates similar to the standard pattern. The sorting region is disposed with respect to each of the plurality of candidates for the standard pattern, extracted in the extraction step, to sort a candidate for the standard pattern based on an evaluation value of the sorting region disposed with respect to each of the plurality of candidates for the standard pattern.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims foreign priority based on Japanese PatentApplication No. 2015-228728, filed Nov. 24, 2015, the contents of whichis incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a positioning method for capturing animage of a product to position the product, a positioning apparatus, aprogram, and a computer readable recording medium.

2. Description of Related Art

Visual inspection is performed for making a pass/fail determination on aproduct manufactured in a factory. In the visual inspection, aninspection target is photographed by a camera to acquire an inspectiontarget image. From a standard image obtained by capturing an image of anon-defective product to be a standard for the product, a region is setso as to surround the non-defective product. A feature (standardpattern, etc.) of the non-defective product is extracted from theregion, and a candidate similar to this feature is searched in aninspection target image. The visual inspection is then performed on thecandidate found by the search. Edge information extracted from thestandard image is often used in order to find the candidate (UnexaminedJapanese Patent Publication No. 2010-067246).

There may be a large number of candidates similar to the standardpattern in the inspection target image. Some candidates may be simplysimilar as images, and thus an extracted candidate may not be a targetof the visual inspection. Also in an application other than the visualinspection, a candidate may need to be sorted.

There is, for example, a request for sorting a product, easy forrobot-picking by a robot hand, out of a plurality of products. Forexample, a product with an obstacle present therearound and likely tofail in robot-picking is desirably removed from the candidates. Further,in an application where an alignment mark provided on a product isdetected to position the product, it is necessary to find a way not toerroneously detect a portion having a similar shape to that of thealignment mark. Accordingly, it is an object of the present invention toaccurately position a feature portion desired by a user out of aninspection target image.

SUMMARY OF THE INVENTION

The present invention, for example, provides a positioning method forsearching a previously registered standard pattern from an inspectiontarget image obtained by capturing an image of an inspection target, toposition the standard pattern with respect to the inspection targetimage. The method includes: a setting step of displaying a standardimage of a product to be a standard for the inspection target, to set afirst region so as to surround the standard pattern in the standardimage, and setting a second region that is a region for sorting aplurality of candidates similar to the standard pattern in theinspection target image; an extraction step of searching the standardpattern from the inspection target image to extract a plurality ofcandidates similar to the standard pattern; a sorting step of disposingthe second region with respect to each of the plurality of candidatesfor the standard pattern, extracted in the extraction step, to sort acandidate for the standard pattern based on an evaluation value of thesecond region disposed with respect to each of the plurality ofcandidates for the standard pattern; and an output step of outputtingthe candidate for the standard pattern sorted in the sorting step.

According to the present invention, it is possible to accuratelyposition a feature portion desired by a user out of an inspection targetimage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view showing a simple overview of a visualinspection apparatus;

FIG. 2 is a diagram showing an example of a hardware configuration ofthe visual inspection apparatus;

FIG. 3 is a flowchart showing a basic flow of visual inspectionprocessing;

FIGS. 4A to 4C are views for explaining the relation between a standardpattern and candidates;

FIGS. 5A to 5C are views for explaining the relation between a standardpattern and candidates;

FIGS. 6A to 6C are views for explaining examples of a standard regionand a sorting region;

FIG. 7 is a view for explaining an example of the standard region andthe sorting region;

FIG. 8 is a diagram for explaining functions of a CPU and an imageprocessing section;

FIG. 9 is a view showing an example of a setting user interface;

FIG. 10 is a flowchart showing setting processing;

FIG. 11 is a flowchart showing an application including sortingprocessing;

FIG. 12 is a flowchart showing the sorting processing;

FIG. 13 is a view showing an example of placement of the standard regionand the sorting region in an inspection image; and

FIG. 14 is a flowchart showing evaluation value deciding processing.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, one embodiment of the present invention will be shown.Individual embodiments described below may be useful for understanding avariety of concepts such as a superordinate concept, an intermediateconcept, and a subordinate concept of the present invention. Further, atechnical range of the present invention is defined by the claims, andis not limited by the following individual embodiments.

FIG. 1 is a schematic view showing a simple overview of a visualinspection apparatus 1. The visual inspection apparatus 1 includes acontroller 2, a programmable logic controller (PLC) 3, a camera 4, anilluminating device 5, a console 9, a monitor 10, and a program creationsupport device 11. A conveyance device 7, such as a belt conveyor,controlled by the PLC 3 carries an inspection target 8. The camera 4captures an image of the inspection target 8 illuminated by theilluminating device 5. The inspection target 8 may also be referred toas a workpiece. For example, in accordance with an order from the PLC 3,the controller 2 switches an illuminating condition for the illuminatingdevice 5, or causes the camera 4 to perform imaging.

The controller 2 performs a variety of measurement processing such asedge detection and area calculation from the image of the inspectiontarget 8. For example, the controller 2 performs image processing by useof image data obtained from the camera 4 and outputs a determinationsignal, as a signal indicating a determination result such as pass/failof the inspection target 8, to externally connected control equipmentsuch as the PLC 3.

The camera 4 is provided with a camera module having an imaging elementfor capturing the image of the inspection target 8. As the imagingelement, for example, a CMOS (Complementary Metal Oxide Semiconductor)or a CCD (Charge-Coupled Device) can be used. The camera 4 captures theimage of the inspection target 8 based on a control signal inputted fromthe PLC 3, such as an imaging trigger signal to define the timing forfetching image data from the camera 4.

The monitor 10 is a display device such as a liquid crystal panel or alight-emitting panel. It displays the captured image of the inspectiontarget 8, and a result of measurement processing performed using theimage data. The monitor 10 may display an image acquired from anon-defective product, such as a reference image to be used for creatingcomparison data (standard image) for pattern matching. Note that thestandard image may be referred to as a model image.

The console 9 is an input device for the user to perform a variety ofoperation on the monitor 10 (the console 9 can be omitted when themonitor 10 is a touch panel). The console 9 selects each menu item andsets a parameter value on the monitor 10. The console 9 is an example ofa pointing device. By viewing the monitor 10, the user can confirm anoperating state of the controller 2 that is running. By operating theconsole 9 while viewing the monitor 10, the user can perform a varietyof setting and edition according to the need.

The illuminating device 5 is a device for illuminating the inspectiontarget 8. As the illuminating device 5, there can be employedilluminating devices for performing a variety of illumination, such ascoaxial vertical illumination for emphasizing a gloss, low-angleillumination for emphasizing an edge of a scratch or a mark, black-lightillumination for throwing black light, surface illumination(trans-illumination for observing transmitted light or a shade of aninspection target), and dome illumination for applying diffused lightfrom all directions. Particularly, the coaxial vertical illumination isan illumination technique of illuminating the entire visual field in asubstantially uniform manner. This illumination has an advantage ofobtaining almost the same effect as that of an illumination technique ofreceiving regularly reflected light from the inspection target 8, bydisposing the camera 4 and the illuminating device 5 in a V-shape.Further, the low-angle illumination is an illumination technique ofdisposing a light projecting element such as an LED in a ring shape andilluminating the surface of the inspection target 8 with light at ashallow angle from all circumferential directions. The light applied onthe surface of the inspection target 8 is not reflected in a directionto the camera 4, and only the light reflected at an edge portion of themark or the scratch is received. That is, with the angle of illuminationvery shallow, the reflection is weak on the gloss surface, and thereflection is strong only at the edge of a tiny scratch on theinspection target 8, whereby clear contrast is obtained.

The program creation support device 11 is a computer (PC) for creating acontrol program to be run by the controller 2. The control program has aplurality of measurement processing modules for performing differenttypes of measurement concerning the visual inspection as describedbelow. The controller 2 calls and runs a variety of measurementprocessing modules in accordance with a set sequence. The programcreation support device 11 is connected with the controller 2 through acommunication cable or a communication network. The setting informationsuch as the control program and the parameter value generated on theprogram creation support device 11 is transmitted to the controller 2through the communication cable or the like. In reverse, the settinginformation, such as the control program and the parameter value, may befetched from the controller 2, to be re-edited by the program creationsupport device 11.

In a factory, a plurality of inspection targets 8 flow on a line of theconveyance device 7 such as a conveyor. The controller 2 captures theimage of the inspection target 8 by the camera 4 installed above (orlateral to, or below) the inspection target 8, and compares the capturedimage with a reference image (e.g., a captured image of a non-defectiveproduct) or a model image created from the reference image, to determinewhether or not a scratch, a defect, or the like exists on the inspectiontarget 8. When it is determined that a scratch, a defect, or the likeexists on the inspection target 8, a fail determination is made. On theother hand, when it is determined that a scratch, a defect, or the likedoes not exist on the inspection target 8, a pass determination is made.As thus described, the visual inspection apparatus 1 makes a pass/faildetermination on appearance of the inspection target 8 by use of thecaptured image of the inspection target 8.

When the visual inspection is to be performed on the inspection target8, the user needs to set details (parameter values, etc.) of a varietyof parameters for use in the inspection. The parameters, for example,include an imaging parameter for defining an imaging condition such as ashutter speed, an illuminating parameter for defining an illuminatingcondition such as an illuminance, a measurement processing parameter(so-called inspection parameter) for defining an inspection conditionshowing what kind of inspection is to be performed, and some otherparameter. In the visual inspection apparatus 1, details of the varietyof parameters are set before the pass/fail determination is made.

The visual inspection apparatus 1 has: a mode for actually performingthe visual inspection on the inspection target 8 flowing one afteranother on the line of the conveyance device 7, namely, an operatingmode (Run mode) for actually making the pass/fail determination on theinspection target 8; and a setting mode (Non-Run mode) for settingdetails of a variety of parameters for use in the inspection. The visualinspection apparatus 1 has a mode switching unit for switching thesemodes. The user sets (adjusts) an optimum parameter value for each of avariety of parameters in the setting mode before the pass/faildetermination is repeatedly made on a plurality of inspection targets 8flowing on the line of the conveyance device 7 in the operating mode.Basically, default values are set for a variety of parameters. When theuser determines the default value as optimum, adjustment of theparameter value is not particularly required. However, in reality, whenthe default value remains unchanged, a determination result desired bythe user may not be obtained due to a surrounding illuminatingenvironment, the installed position of the camera 4, postural deviationof the camera 4, a difference in focus adjustment, or the like.Accordingly, in the setting mode, the mode can be switched from theoperating mode to the setting mode on the monitor 10 or the programcreation support device 11 of the controller 2, to edit details of avariety of parameters.

<Hardware Configuration of Visual Inspection Apparatus 1>

FIG. 2 is a diagram showing an example of a hardware configuration ofthe visual inspection apparatus 1. A main control section 21 controlseach section of the hardware while performing numeric calculation andinformation processing based on a variety of programs. For example, themain control section 21 includes: a CPU 22 as an intermediate operationprocessing device; a working memory 23 such as a RAM which functions asa working area at the time of running a variety of programs; and aprogram memory 24 such as a ROM, a flash ROM or an EEPROM which stores astartup program, an initialization program, and the like. Anillumination controlling section 26 transmits an illumination controlsignal to the illuminating device 5 based on an order from the CPU 22 ofthe main control section 21 or from the PLC 3.

An image inputting section 25 is made up of an ASIC (ApplicationSpecific Integrated Circuit), which fetches image data acquired byimaging by the camera 4, and the like. The image inputting section 25may include a frame buffer for buffering image data. Specifically, whenreceiving an imaging command for the camera 4 from the CPU 22, the imageinputting section 25 transmits an image data fetching signal to thecamera 4. After the imaging by the camera 4, the image inputting section25 fetches image data obtained by the imaging. The fetched image data isonce buffered (cached).

An operation inputting section 27 receives an input of an operationsignal from the console 9. The operation inputting section 27 functionsas an interface (I/F) for receiving an operation signal that isoutputted by the console 9 in accordance with the user's operation.

The monitor 10 displays details of the user's operation performed usingthe console 9. Specifically, by operating the console 9, the user can doa variety of things on the monitor 10, such as editing a control programfor image processing, editing a parameter value for each measurementprocessing module, setting imaging conditions for the camera 4,registering a characteristic portion in a reference image as a standardimage, and performing search in a search region to set a region matchingwith the standard image as an inspection region.

A display controlling section 28 is made up of a DSP for display whichdisplays an image on the monitor 10, and the like. The displaycontrolling section 28 may include a video memory such as a VRAM fortemporarily storing image data at the time of displaying an image. Basedon a display command transmitted from the CPU 22, the displaycontrolling section 28 transmits a control signal for displaying apredetermined image (video image) on the monitor 10. For example, thedisplay controlling section 28 transmits a control signal to the monitor10 in order to display image data before measurement processing or aftermeasurement processing. Further, the display controlling section 28 alsotransmits a control signal for displaying on the monitor 10 details ofthe user's operation performed using the console 9.

A communication section 29 is connected communicably with the externalPLC 3 and program creation support device 11, and the like. For example,a sensor (photoelectric sensor, not shown, etc.) is installed on amanufacturing line in order to recognize the arrival timing of theinspection target 8, and is also connected to the PLC 3. The sensoroutputs a trigger signal when detecting the inspection target 8. Thecommunication section 29 functions as an interface (I/F) for receivingthe trigger signal outputted from the PLC 3. The trigger signal is usedas a control signal for causing the camera 4 to perform imaging. Thecommunication section 29 also functions as an interface (I/F) forreceiving a control program for the controller 2, and the like,transmitted from the program creation support device 11.

An image processing section 30 is made up of a DSP for calculation,which performs measurement processing such as edge detection and areacalculation, and the like. The image processing section 30 may include amemory for storing image data for measurement processing. The imageprocessing section 30 performs measurement processing on image data.Specifically, the image processing section 30 reads image data from aframe buffer of the image inputting section 25, to perform internaltransmission to the memory in the image processing section 30. The imageprocessing section 30 then reads the image data stored in the memory, toperform measurement processing.

The program memory 24 stores a control program for controlling each ofthe illumination controlling section 26, the image inputting section 25,the operation inputting section 27, the display controlling section 28,the communication section 29, and the image processing section 30, byuse of a command of the CPU 22 and the like. Further, the controlprogram transmitted from the program creation support device 11 isstored into the program memory 24.

When the CPU 22 receives the imaging trigger signal from the PLC 3through the communication section 29, it transmits an imaging command tothe image inputting section 25. Further, based on the control program,the CPU 22 transmits to the image processing section 30 a command thatinstructs image processing to be performed. As the device to generatethe imaging trigger signal, the PLC 3 may not be used, but a triggerinputting sensor such as a photoelectric sensor may be directlyconnected to the communication section 29.

The hardware described above is communicably connected with each otherthrough electric communication path (cable) such as a bus.

<Measurement Module (Image Processing Tool)>

Here, a measurement module for performing the visual inspection isreferred to as an image processing tool. Note that the image processingtool may also be referred to as an inspection tool or a measurementtool. There are a variety of image processing tools, and main imageprocessing tools include an edge position measuring tool, an edge anglemeasuring tool, an edge width measuring tool, an edge pitch measuringtool, an area measuring tool, a blob measuring tool, a pattern searchmeasuring tool, a scratch measuring tool, and the like.

-   -   Edge position measuring tool: On the screen that displays the        image of the inspection target 8, a window is set with respect        to an inspection region having an edge position to be detected,        and scanning is performed in an arbitrary direction in the set        inspection region, to detect a plurality of edges (a place where        the brightness changes from bright to dark or a place where the        brightness changes from dark to bright). Specification of one        edge from the plurality of detected edges is accepted, and a        position of the accepted edge is then measured.    -   Edge angle measuring tool: In the inspection region accepted to        be set, two segments are set to measure an angle of inclination        of the inspection target 8 from the edge detected in each of the        segments. As for the angle of inclination, a clockwise direction        can be taken as positive, for example.    -   Edge width measuring tool: In the inspection region accepted to        be set, scanning is performed in an arbitrary direction to        detect a plurality of edges and measure a width between the        plurality of detected edges.    -   Edge pitch measuring tool: In the inspection region accepted to        be set, scanning is performed in an arbitrary direction to        detect a plurality of edges. The maximum/minimum values and an        average value of distances (angles) between the plurality of        detected edges are measured.    -   Area measuring tool: Binarization processing is performed on the        image of the inspection target 8 captured by the camera 4, to        measure an area of a white region or a black region. For        example, specification of the white region or the black region        as a measurement target is accepted as a parameter, to measure        the area of the white region or the black region.    -   Blob measuring tool: Binarization processing is performed on the        image of the inspection target 8 captured by the camera 4, to a        measure a number, an area, a centroid position, and the like as        parameters with respect to a set (blob) of pixels with the same        luminance value (255 or 0).    -   Pattern search measuring tool: An image pattern (model image) to        be a comparison target is previously stored in a storage device,        and a portion of the captured image of the inspection target 8,        which is similar to the stored image pattern, is detected to        measure a position, an angle of inclination, and a correlation        value of the image pattern.    -   Scratch measuring tool: In the inspection region accepted to be        set, a small region (segment) is moved to calculate an average        concentration value of pixel values and determine a position        with a concentration difference not smaller than a threshold as        having a scratch.    -   Other than the above, there are tools such as: an OCR        recognition tool for cutting out character information in the        inspection region and checking it with dictionary data or the        like to recognize a character string; a trend edge tool having a        function of shifting a window (region) set on the screen and        repeat edge detection in a position of each window; and a        gradation tool having a function to measure an average, a        deviation, and the like of gradations in the set window. The        user can select a required image processing tool according to        details of the inspection. Note that these image processing        tools merely show typical functions and representative examples        of the method for realizing the functions. An image processing        tool corresponding to any image processing can be a target of        the present invention.

<Basic Flow of Visual Inspection>

FIG. 3 is a flowchart showing a basic flow of visual inspectionprocessing. The visual inspection processing is divided into a settingmode and an operating mode. The setting mode is a mode for setting astandard image, an inspection region, a search region, a detection point(hereinafter referred to as reference point), a reference line, and athreshold such as a tolerance, which are required for making thepass/fail determination on the inspection target 8. The operation modeis a mode for actually capturing the image of the inspection target 8and performing image processing such as pattern matching to make thepass/fail determination. For appropriately setting parameters forinspection, it is common that the setting mode and the operating modeare repeatedly performed. Note that the visual inspection processing mayalso be performed by size measurement, an area tool, a scratch tool, orthe like.

In S1, the CPU 22 transmits an imaging order to the camera 4 through theimage inputting section 25, to cause the camera 4 to perform imaging.The CPU 22 displays image data acquired by the camera 4 on the monitor10 through the display controlling section 28. By viewing the imagedisplayed on the monitor 10, the user confirms the posture of the camera4 and the illuminating state of the illuminating device 5.

In S2, the CPU 22 adjusts an exposure condition such as a shutter speedof the camera 4 based on an instruction inputted by the console 9. Notethat the user may manually adjust the posture of the camera 4.

In S3, the CPU 22 transmits an imaging order to the camera 4 in order tofetch as a workpiece image the image of the inspection target 8 that isdisposed in an imaging position of the conveyance device 7. Note thatthe workpiece image (basic image) may be the reference image that isstored in a nonvolatile memory and repeatedly used, or may be an imagethat is captured each time for creating the standard image. Here, theworkpiece image is stored in the working memory 23. Note that the modelimage may be created from the reference image.

In S4, the CPU 22 performs setting processing for position/posturecorrection. In the image acquired by the camera 4, the position of theimage of the inspection target 8 may be deviated from an ideal position.

Therefore, the CPU 22 acquires the position and posture (rotating angle)of the inspection target 8 from the image of the inspection target 8.The position and posture of the image processing tool (inspection tool)are corrected according to the acquired position, to correct thepositional deviation. Note that the position/posture correction may beperformed by the image processing section 30. As thus described, theposition/posture correction is processing for matching the position andposture of the inspection tool with the position and posture of theimage of the inspection target 8.

In S5, the CPU 22 sets a variety of inspection tools described above.For example, there are made a setting for which measurement is to beperformed in the visual inspection, and settings for a search region, aninspection region, a reference point, and the like, which are requiredfor performing the measurement.

In S6, the CPU 22 sets a parameter required in the visual inspection(e.g., inspection threshold such as tolerance) based on an instructioninputted by the console 9. In S7, the CPU 22 switches the mode from thesetting mode to the operating mode.

In S8, the CPU 22 causes the camera 4 to capture the image of theinspection target 8 in accordance with an instruction from the PLC 3,and causes the image processing section 30 to perform pattern matchingor the like. Based on its result, the CPU 22 makes the pass/faildetermination, and outputs the determination result to the PLC 3 and tothe monitor 10.

In S9, when the CPU 22 receives an input of a mode switching instructionfrom the console 9, it switches the mode from the operating mode to thesetting mode. In S10, the CPU 22 re-sets a parameter based on aninstruction inputted by the console 9.

<Basic Idea of Sorting Technique>

FIG. 4A shows a standard pattern 31 of an alignment mark provided onpart of the inspection target 8. The standard pattern is a feature of animage extracted by image processing from a standard image (also referredto as a model image or a template image) obtained by capturing an imageof a non-defective product of the inspection target 8. The alignmentmark is used for positioning the inspection target 8 with respect to aprocessing machine so as to process the inspection target 8.

FIG. 4B shows an alignment mark 32 formed on the inspection target 8,and a similar shape 33 partially similar to the alignment mark 32. Thereare cases where the alignment mark 32 is scratched, or where an image ofthe alignment mark 32 is unclearly taken at the time of capturing animage of the inspection target 8, due to an influence of illuminatinglight. In such cases, part of the similar shape 33 may be detected as acandidate for the standard pattern 31. Further, in the case of selectingthe candidate based on an evaluation value obtained by an edge matchingdegree, so long as an edge similar to the standard pattern 31 is presentin the inspection image, the evaluation value with respect to thestandard pattern 31 becomes undesirably high. That is, even when anotheredge continuing to an edge similar to the standard pattern 31 is presentin the inspection image, the evaluation value becomes undesirably highin the technique relying on the evaluation value. When a scratch, amark, or the like inside the standard pattern 31 does not form a contouron the standard pattern, or when the mark or the like, even if forming acontour, is relatively small as compared with the standard pattern, itmay not be possible to sort out the candidate for the standard pattern31 by the technique relying on the evaluation value of the contour.There has been described the example of the evaluation value being sucha value as to become high when an edge constituting the standard patternis compared with an edge on the inspection image and an edgecorresponding to the edge constituting the standard pattern is presenton the inspection image. However, a variety of techniques can beemployed as a technique of calculating the evaluation value. Forexample, a correlation value obtained by normalized correlation or thelike may be employed. A technique of focusing attention on normalizedcorrelation of an image has an advantage in that a correlation valuebecomes low when there is a portion not present in the standard image.However, this technique has the same problem as the edge evaluation inthe following respect: even with the use of the correlation valueobtained by the normalized correlation, when the inspection image isunclear or when the mark or the like is relatively small as comparedwith the standard pattern, the presence or absence of the mark or thescratch cannot be discriminated just by the correlation value. Note thatan evaluation value representing a similarity to the standard pattern,including the evaluation value obtained by the edge matching degree andthe correlation value obtained by the normalized correlation, is simplyreferred to as a correlation value.

FIG. 4C shows an example of only sorting a face-up workpiece(semiconductor chip 36 a) out of a plurality of workpieces(semiconductor chips 36 a, 36 b). There is a request for mounting asmall electronic component face-up at the time of mounting the smallelectronic component on a substrate. When the electronic component issearched based on an external feature of the standard pattern as hasbeen done, a face-down electronic component as well as a face-upelectronic component is undesirably detected as the candidate.

FIG. 5A shows a screw 35 being a target for robot-picking by a robothand. By photographing the screw 35, a standard pattern of the screw 35is extracted. FIG. 5B show a plurality of screws 35 a, 35 b having beencarried by the belt conveyor or the like. For grasping the screw 35 bythe robot hand, no obstacle is desirably present around the screw 35.For example, since other screws are present around the screw 35 a,performing the robot-picking on the screw 35 a by the robot is not easy.Hence it is desirable to detect as a candidate a screw 35 b with noobstacle present therearound. However, when search is performed usingthe standard pattern extracted from the screw 35, as shown in FIG. 5C,all screws 35 similar to the standard pattern are undesirably extractedas candidates.

As described above, in each of a variety of applications such as thevisual inspection and robot-picking, there is required a technique ofsorting a workpiece more suitable for a purpose of the application amonga plurality of candidates similar to the standard pattern.

In the present embodiment, a sorting region is set with respect to thestandard image in order to sort a desired candidate out of a pluralityof candidates similar to the standard pattern. Further, a standardregion for extracting the standard pattern is also set in the standardimage. Note that the positional relation between the standard patternand the sorting region is set using the standard image, and thispositional relation is also held in the inspection target image. Evenscaling processing is intervened, these positional relations are held.

FIG. 6A shows a placement example of a standard region 40 for setting orextracting an outline (contour) of an alignment mark as the standardpattern 31, and sorting regions 41 for sorting a candidate for thestandard pattern 31 in an inspection target image (inspection image). Inthis example, since a plurality of sorting regions 41 are provided, thesorting region 41 disposed on the left side of the standard region 40 isreferred to as a sorting region 41 a, and the sorting region 41 disposedon the right side of the standard region 40 is referred to as a sortingregion 41 b. As thus described, characters a, b, and the like are justexpediential characters that are provided for distinguishing a pluralityof sorting regions 41. When a matter in common among the plurality ofsorting regions 41 is to be described, the characters a, b, and the likeare omitted. The standard region 40 and the sorting region 41 are setwith respect to the standard image, and a relative positional relationbetween the standard region 40 and the sorting region 41 is held. Notethat a detecting point 42 is set in the standard image.

FIG. 6B shows a placement example of the standard region 40 for settingor extracting a contour of the screw 35 as the standard pattern 31, andsorting regions 41 for sorting a candidate for the standard pattern 31in the inspection image. Since the robot picking is assumed to beperformed on the screw 35 in this example, the sorting regions 41 a, 41b are provided in a working range for the robot-picking.

FIG. 6C shows a placement example of the standard region 40 for settingor extracting a contour of the semiconductor chip 36 as the standardpattern 31, and a sorting region 41 for sorting a candidate for thestandard pattern 31 in the inspection image. Since an application formounting the semiconductor chip 36 face-up is assumed in this example,the sorting region 41 for sorting the face-up semiconductor chip 36 aand the face-down semiconductor chip 36 b is provided. In this example,characters such as an identification number is provided on the frontsurface of the semiconductor chip 36, and no characters are provided onthe rear surface thereof.

Therefore, when the sorting region 41 is set with respect to thecharacters of the front surface, the front and the rear of thesemiconductor chip 36 can be distinguished.

FIG. 7 is a view showing an example of a result of sorting by use of thesorting regions. By using a contour feature, extracted from the standardregion 40, as the standard pattern 31, a plurality of candidates similarto the standard pattern 31 are extracted as shown in FIG. 5C. Next,attention is focused on the two sorting regions 41 a, 41 b as shown inFIG. 6B. In this example, the two sorting regions 41 a, 41 b are bothset as having no obstacle as a sorting condition. Of a plurality ofcandidates extracted from the inspection image, a feature of an imageand the sorting condition in each of the two sorting regions 41 a, 41 bare compared. As a result, as shown in FIG. 7, the sorting regions 41 a,41 b for the screw 35 a and the like do not satisfy the sortingcondition, and hence the screw 35 a and the like are excluded from thecandidates. The sorting regions 41 a, 41 b for the screw 35 b satisfythe sorting condition, and hence the screw 35 b is decided as a finalcandidate.

<Functions of CPU and Image Processing Section>

FIG. 8 is a diagram showing an example of functions of the CPU 22 andthe image processing section 30. The CPU 22 runs control programs storedin the program memory 24, to realize a variety of functions. Part or allof these functions may be realized by a logic circuit such as an ASIC oran FPGA.

A UI control section 50 controls a user interface (UI) for performing avariety of settings required in performing a variety of applicationssuch as the visual inspection. The UI control section 50 accepts theuser's instruction inputted from the console 9 or the like, and displaysinformation on the monitor 10 through the display controlling section28. For example, when a position correction setting is instructedthrough the console 9, the UI control section 50 displays on the monitor10 a position correction setting UI 80 as shown in FIG. 9. The positioncorrection setting UI 80 includes: a display region 81 for displaying astandard image, a check box 82 for enabling/disabling the sortingprocessing; an edition button 83 for editing the standard region 40; anedition button 84 for editing the sorting region 41; an edition button85 for editing the sorting condition; and the like. When the check box82 is checked, the sorting enabling section 54 stores, in the settingdata storing section 62, information indicating that the sortingprocessing has been enabled. When the check box 82 is checked, the UIcontrol section 50 may change the state of each of the edition button 84for editing the sorting region 41 and the edition button 85 for editingthe sorting condition from an inoperable state (graying out, etc.) to anoperable state.

A standard region setting section 52 sets the standard region 40 withrespect to the standard image displayed in the display region 81 inaccordance with the user's instruction from the console 9. The standardregion setting section 52 stores, in the setting data storing section62, data indicating a position (e.g., coordinates of four vertexes,coordinates of a detecting point 42, etc.) of the standard region 40 inthe standard image. Note that the image processing section 30 mayextract a feature of the standard pattern 31 surrounded by the standardregion 40, and store data indicating the feature in the setting datastoring section 62. When a search technique without the need forextracting the feature of the standard pattern 31 is to be employed, thefeature extracting processing performed in advance is omitted.

A sorting region setting section 53 sets the sorting region 41 withrespect to the standard image displayed in the display region 81 inaccordance with the user's instruction from the console 9. The sortingregion setting section 53 stores, in the setting data storing section62, data indicating the position (e.g., a distance from the origin orthe detecting point 42 to each of four vertexes, etc.) of each sortingregion 41 with respect to the standard region 40. Note that the imageprocessing section 30 may extract a feature of part of the standardimage surrounded by the sorting region 41, and store data indicatingthis feature as the sorting condition in the setting data storingsection 62. When a sorting technique without the need for extracting thefeature of the sorting region 41 is to be employed, the featureextracting processing performed in advance is omitted.

A sorting condition setting section 63 stores a sorting condition, setby the user operating the edition button 85 for the sorting condition,in the setting data storing section 62. For example, an upper limit anda lower limit of the number of edge pixels or a ratio of edge pixels maybe set as the sorting conditions. Since the thresholds such as the upperlimit and the lower limit depend on the kind of the inspection target 8or a peripheral situation of the standard pattern 31, the user mayrepeat a test or the like to find appropriate values.

The inspection tool selecting section 55 selects the inspection tool tobe performed on the inspection target 8 in accordance with the user'sinstruction inputted through the console 9. The visual inspectionsection 68 of the image processing section 30 has a variety ofinspection tools, but all of those are not always necessary. The userselects the inspection tool to be performed according to the inspectiontarget 8. An inspection region setting section 56 sets a region(inspection region) to be the inspection target by the inspection toolselected by the inspection tool selecting section 55. For example, whena measurement tool for measuring a distance from a first part to asecond part of the inspection target 8 is selected, an inspection regionsurrounding the first part and an inspection region surrounding thesecond part are set with respect to an image of a product (inspectionpassed product) to be a standard for the inspection target 8. Forexample, when a measurement tool for measuring an area of the first partof the inspection target 8 is selected, an inspection region surroundingthe first part is set with respect to the image of the product(inspection passed product) to be the standard for the inspection target8. The inspection threshold setting section 57 sets an inspectionthreshold to be a reference for making the pass/fail determination onthe inspection target 8. For example, when a measurement tool formeasuring a distance is selected by the inspection tool selectingsection 55, the inspection threshold setting section 57 sets as theinspection threshold a range (tolerance) to be a pass criterion for thedistance. The inspection threshold is inputted by the user through theconsole 9 or the like. These setting data are also stored in the settingdata storing section 62, and in the operating mode, the setting data isread by the visual inspection section 68 and used. Note that eachinspection region may be associated with the standard region. That is,the position of each inspection region may be previously decided withthe position of the standard region taken as a reference. That is, thestandard region may also be used as the position correcting region.

In the setting mode, an imaging control section 58 controls the camera 4and the illuminating device 5 to capture an image of the product(inspection passed product/non-defective product/reference product/modelproduct) to be the standard for the inspection target 8, and storesimage data of the standard image in a standard image storing section 60.Further, in the operating mode, the imaging control section 58 controlsthe camera 4 and the illuminating device 5 to capture the image of theinspection target 8 (uninspected product) to acquire an inspectionimage, and stores image data of the inspection image in an inspectionimage storing section 61. A pass/fail determination section 59 comparesthe result of the visual inspection, received from the visual inspectionsection 68, with the threshold set by the inspection threshold settingsection 57, to make a determination as to whether or not the inspectiontarget 8 present in the inspection image is a non-defective product.

The image processing section 30 also include a variety of functions. Acandidate extracting section 64 extracts a feature of the standardpattern 31 from the standard image, and extracts a candidate for thestandard pattern 31, similar to the extracted feature, from theinspection image. A candidate sorting section 65 sorts a candidatematching with the sorting condition out of a plurality of candidatesextracted from the inspection image. An output section 66 outputs thesorting result to the position/posture deciding section 67, the visualinspection section 68, or the monitor 10.

The position/posture deciding section 67 decides the inspection regionof the inspection tool aligned with the position of the sortedcandidate. A visual inspection section 68 performs the visual inspectionby use of the inspection tool selected by an inspection tool selectingsection 55. When just the robot-picking is performed and the visualinspection is not performed, the function concerning the visualinspection may be omitted. In this case, the position/posture decidingsection 67 may control the position of the robot-picking based on theposition of the candidate.

<Setting Processing>

FIG. 10 is a flowchart showing setting processing including a settingfor the sorting region 41, and the like. In S11, the CPU 22 (imagingcontrol section 58) controls the camera 4 and the illuminating device 5to capture an image of the product to be the standard for the inspectiontarget 8, and stores image data of the standard image in the standardimage storing section 60. The standard image may only have to be held inthe period when the CPU 22 has shifted to the setting mode, but thestandard image may be continuously held even in the period when the CPU22 has shifted to the operating mode. In S12, the CPU 22 (UI controlsection 50) reads image data of the standard image from the standardimage storing section 60, and displays it on the monitor 10 through thedisplay controlling section 28. As described using FIG. 9, the UIcontrol section 50 displays the position correction setting UI 80, andparticularly displays the standard image in the display region 81.

In S13, the standard region setting section 52 sets the standard region40 with respect to the standard image displayed in the display region 81in accordance with the user's instruction from the console 9. The usersets the standard region 40 so as to surround the standard pattern 31included in the standard image. The standard region setting section 52stores, in the setting data storing section 62, a position (e.g.,coordinates of four vertexes, coordinates of a detecting point 42, etc.)of the standard region 40 in the standard image. In S14, the imageprocessing section 30 extracts the feature of the standard pattern 31surrounded by the standard region 40, and stores data indicating thefeature in the setting data storing section 62. As described above, S14may be omitted depending on the search technique.

In S15, the sorting region setting section 53 sets the sorting region 41with respect to the standard image displayed in the display region 81 inaccordance with the user's instruction from the console 9. The positionof the sorting region 41 is different according to the application. Forexample, in the robot-picking, as shown in FIG. 6B, the sorting regions41 are disposed in a region where a picking section is operated. In thecase of the alignment mark, as shown in FIG. 6A, the sorting regions 41are disposed on the right and left of the alignment mark. In the case ofdistinguishing the front and the rear of the workpiece, as shown in FIG.6C, the sorting region 41 is disposed in a portion helpful fordistinguishing the front and the rear. The sorting region settingsection 53 stores, in the setting data storing section 62, dataindicating the position (e.g., a distance from the origin or thedetecting point 42 to each of four vertexes, etc.) of each sortingregion 41 with respect to the standard region 40. Note that the positionof each sorting region 41 may be expressed by a vector. For example,this vector may be a vector indicating a distance and a direction from areference point of the standard region to a reference point of thesorting region. Further, when the standard region and the sorting regionare not restricted to a rectangular shape, shape information indicatingthe shapes of these regions may be stored.

In S16, the sorting condition setting section 63 stores, in the settingdata storing section 62, a sorting condition set by the user operatingthe edition button 85 for the sorting condition. For example, when anevaluation value is to be used for sorting, a threshold of theevaluation value or the like are set as the sorting condition.

<Positioning Processing>

FIG. 11 is a flowchart showing positioning processing that includes astep of sorting out of a plurality of candidates. In S20, the CPU 22(imaging control section 58) controls the camera 4 and the illuminatingdevice 5 to capture the image of the inspection target 8 (uninspectedproduct), and stores image data of the inspection image in theinspection image storing section 61. Depending on the application, oneinspection image may include a plurality of workpieces, or oneinspection image may include one workpiece.

In S21, the image processing section 30 (candidate extracting section64) searches the feature, extracted from the standard region 40 of thestandard image, out of the inspection image to extract a candidate forthe standard pattern 31. Hence the position of the candidate for thestandard pattern 31 in the inspection image is found. When a correlationvalue of the standard pattern 31 and the candidates is lower than apredetermined lower limit, such a candidate may be removed.

In S22, the candidate sorting section 65 sorts a candidate matching withthe sorting condition out of a plurality of candidates extracted fromthe inspection image. Note that more detailed specific examples of thesorting processing will be described later by use of FIG. 12 and thelike.

In S23, the output section 66 outputs the sorting result to at least oneof the position/posture deciding section 67, the visual inspectionsection 68, and the monitor 10. For example, the output section 66 mayprovide visually distinguishable marks respectively on the sortedcandidate and the unsorted candidate of the plurality of candidatesextracted in S21. For example, a red frame may be displayed for thesorted candidate so as to surround the candidate, and a yellow frame maybe displayed for the unsorted candidate so as to surround the candidate.Needless to say, some intensified display may be made for the sortedcandidate so as to surround the candidate, and such an intensifieddisplay may be omitted for the unsorted candidate. The sorting resultmay include putting up a flag (setting a flag bit to 1) for the sortedcandidate, or bringing down a flag (setting a flag bit to 0) for theunsorted candidate. Further, the sorting result may include coordinatesindicating the position of the sorted candidate. The reason fordisplaying the unsorted candidate is because it is helpful in debugging.Whether such a candidate has been rejected based on the threshold of thecorrelation value, or based on the sorting threshold, is informationhelpful in debugging. Therefore, displaying the unsorted candidate (thecandidate rejected based on the sorting threshold) is useful for aperson in charge of debugging.

In S24, the CPU 22 performs processing in accordance with theapplication by use of the sorting result. For example, the CPU 22 movesthe robot hand based on coordinates included in the sorting result, andcauses the robot hand to pick up a screw as the sorted candidate. TheCPU 22 may move the robot hand based on the coordinates included in thesorting result, and pick a face-up electronic component as the sortedcandidate, to position the electronic component in a mountable positionon a printed circuit board. Further, the CPU 22 and the position/posturedeciding section 67 may dispose the inspection region of the inspectiontool in the inspection image according to the coordinates of thealignment mark included in the sorting result, and may perform thevisual inspection on the inspection region.

<Sorting Processing>

FIG. 12 is a flowchart showing several steps that can be included in thesorting processing (S22). Note that when N candidates are found, thesorting processing is performed on each of the N candidates.

In S30, the candidate sorting section 65 disposes the sorting regionaccording to the position of the candidate. At the time when eachcandidate is found, the position of the standard region 40 of eachcandidate in the inspection image has been settled. Further, a relativepositional relation between the standard region 40 and each sortingregion 41 is known, and this is held in the setting data storing section62. Therefore, the candidate sorting section 65 decides the sortingregion 41 concerning each candidate from the position of the standardregion 40 of each candidate in the inspection image and from data of therelative positional relation held in the setting data storing section62.

FIG. 13 shows the positional relation between the standard region 40 andeach of the sorting regions 41 a, 41 b in the standard image 70. In theinspection image 90, the position and posture (rotating angle) of thecandidate for the standard pattern 31 are often shifted with respect tothe standard pattern 31. Thus, the candidate for the standard pattern 31in the inspection image 90 is found, and the sorting regions 41 a, 41 bare disposed with respect to the position of the candidate for thestandard pattern 31 such that the positional relation between thestandard region 40 and each of the sorting regions 41 a, 41 b is held.For example, a transformation matrix is obtained for transforming theposition and the posture of the detecting point 42 set at the center ofthe standard region 40 to a position and posture of a detecting point inthe inspection image 90, and the positions of the sorting regions 41 a,41 b are transformed by use of this transformation matrix to decide thepositions of the sorting regions 41 a, 41 b in the inspection image 90.In the transformation matrix, a change in scaling (magnification) mayalso be considered.

In S31, the candidate sorting section 65 decides an evaluation valueconcerning the image in the sorting region disposed in the inspectionimage 90. A technique of deciding the evaluation value will be describedusing FIG. 14.

In S32, the candidate sorting section 65 compares the sorting conditionand the evaluation value obtained from the sorting region as to each ofthe plurality of candidates, and sorts a candidate having brought theevaluation value that satisfies the sorting condition.

<Method for Deciding Evaluation Value>

There are a large number of indexes that can be employed as evaluationvalues. For example, the evaluation value relying on the sorting region41 set with respect to the inspection image 90 includes the number ofedge pixels, the ratio of edge pixels, an average pixel value, anaverage color difference, an edge position, the number of edges, an edgewidth, and a combination of those. Further, the evaluation valuerelaying on both the sorting region 41 set with respect to theinspection image 90 and the sorting region 41 in the standard image 70includes a contrast ratio, a difference in average color, a differenceabsolute sum, a correlation value; a mutual information amount; and aphase limited correlation value of normalized correlation, a differencein edge position, a difference in edge width, gradation or a distance ofa histogram of an edge strength or an edge angle, and a combination ofthose. Hereinafter, the number of edge pixels and the ratio of edgepixels will be described as an example of the evaluation value.

FIG. 14 is a flowchart showing a several steps that can be included inthe evaluation value deciding processing (S31). In S40, the candidatesorting section 65 obtains a circumscribed rectangle that iscircumscribed to the sorting region 41 disposed with respect to eachcandidate for the standard pattern 31 in the inspection image 90.According to FIG. 13, a circumscribed rectangle 91 a is decided withrespect to the sorting region 41 a, and a circumscribed rectangle 91 bis decided with respect to the sorting region 41 b.

In S41, the candidate sorting section 65 obtains a vertical edge imageand a horizontal edge image concerning images in the circumscribedrectangles 91 a, 91 b. For example, the candidate sorting section 65applies the Sobel filter on the images in the circumscribed rectangles91 a, 91 b, to obtain the vertical edge image and the horizontal edgeimage. In S42, the candidate sorting section 65 obtains an edge strengthimage and an edge angle image from the vertical edge image and thehorizontal edge image. When the processing by direction is to beomitted, generation of an edge angle image is omitted. The circumscribedrectangle is used in order to simplify the processing even when thecandidate sorting region has a complicated shape. However, thecircumscribed rectangle may not be used and the edge extraction may beperformed particularly inside the shape. In this case, S47 describedlater may be omitted.

In S43, the candidate sorting section 65 corrects each pixel of the edgeangle image by the rotating angle of the candidate. A pixel value ofeach pixel constituting the edge angle image indicates an edge angle.Angles formed by the circumscribed rectangles 91 a, 91 b and the sortingregions 41 a, 41 b are the same as an angle formed by the standardpattern 31 in the standard image 70 and the candidate for the standardpattern 31 in the inspection image 90. Each pixel of the edge angleimage is corrected by subtracting this rotating angle from each pixel.That is, the edge angle image is corrected by the rotating angle. Wheneach pixel of the edge angle image, corrected by the rotating angle, isnot within a predetermined edge angle range, the candidate sortingsection 65 may set a pixel value of the pixel in the edge strength imagewhich corresponds to the above pixel to 0. In such a manner, the edgeangle image is used for correcting the edge strength image. Correctingthe edge strength image using the edge angle image can reduce aninfluence of noise. That is, the sorting is performed using a featurewith directional dependency, thereby reducing an influence of noise withlow directional dependency. Note that S43 may be omitted as to aninspection target not required to be subjected to the processing bydirection.

In S44, the candidate sorting section 65 corrects a pixel lower than thelower limit threshold in the edge strength image. For example, thecandidate sorting section 65 changes a pixel value of a pixel to 0, thepixel being lower than the lower limit threshold in the edge strengthimage.

In S45, the candidate sorting section 65 obtains a coordinatetransformation matrix for transforming the coordinate system of thestandard image 70 to the coordinate system of the inspection image 90.For example, the candidate sorting section 65 obtains the coordinatetransformation matrix from the coordinates of the detecting point 42 ofthe standard pattern 31 in the standard image 70 and the coordinates ofthe detecting point 42 concerning the candidate for the standard pattern31 in the inspection image 90. In S46, the candidate sorting section 65obtains an inverse matrix of the coordinate transformation matrix.

In S47, the candidate sorting section 65 obtains a position of a pixelwith its pixel value (edge strength) not being 0 in the circumscribedrectangle, to obtain a corresponding position in the standard image. Forexample, the candidate sorting section 65 performs coordinatetransformation by use of an inverse matrix on the position of the pixelwith its pixel value not being 0 in the circumscribed rectangle, toobtain the corresponding position in the standard image.

In S48, the candidate sorting section 65 determines whether or not thecorresponding position in the standard image is in the sorting region,to obtain the number of corresponding positions in the sorting region asthe number of edge pixels. Note that the candidate sorting section 65may calculate the number of edge pixels standardized by dividing thenumber of corresponding positions by a square of a scaling value.

In S49, the candidate sorting section 65 obtains the ratio of edgepixels from the number of edge pixels. For example, the candidatesorting section 65 may divide the number of edge pixels by the area ofthe sorting region 41 to calculate the ratio of edge pixels. In such amanner, the ratio of edge pixels is calculated as the evaluation value.

<Summary>

According to the present embodiment, there is provided a positioningmethod for searching a previously registered standard pattern out of aninspection target image (inspection image) obtained by capturing animage of the inspection target 8, to position the standard pattern withrespect to the inspection target image. Positioning of the standardpattern is, for example, obtaining a position (coordinates, posture,etc.) of the standard pattern in the inspection image, disposing aninspection region for visual inspection with respect to the standardpattern, controlling a position of a grasping section (robot hand) forrobot-picking, and the like.

According to S12, there is provided a display step of displaying astandard image of a product to be a standard for the inspection target8. According to S13, there is provided a setting step of setting astandard region that is a first region so as to surround the standardpattern in the standard image. According to S15, there is provided asetting step of setting a sorting region that is a second region to be aregion for sorting a plurality of candidates similar to the standardpattern in the inspection image. According to S21, there is provided anextraction step of searching the standard pattern from the inspectionimage to extract a plurality of candidates similar to the standardpattern. According to S22, there is provided a sorting step of disposingthe sorting region with respect to each of the plurality of candidatesfor the standard pattern, extracted in the extraction step, to sort acandidate for the standard pattern based on an evaluation value of thesorting region disposed with respect to each of the plurality ofcandidates for the standard pattern. According to S33, there is providedan output step of outputting a candidate for the standard pattern sortedin the sorting step. As described above, according to the presentembodiment, the sorting region is provided separately from the standardregion, to thereby allow accurate positioning of a feature portiondesired by the user out of the inspection image.

As described concerning S32, the sorting step may include adetermination step of determining whether the evaluation value of thesorting region disposed with respect to each of the plurality ofcandidates for the standard pattern satisfies a predetermined condition.Further, the determination step may include a step of determiningwhether the evaluation value of the sorting region concerning theplurality of candidates for the standard pattern, extracted from theinspection image, exceeds a threshold. As thus described, the candidatesmay be narrowed down by use of the threshold related to the number ofedge pixels, or the like.

As described concerning S21, the extraction step may include a step ofextracting a plurality of candidates similar to the standard patternbased on a correlation value of the standard pattern and the inspectionimage. For example, the standard pattern is searched in the inspectionimage while coordinates, a position, a scale (Affine parameters) or thelike of the standard pattern is changed with respect to the inspectionimage, to decide as the candidate a partial image where the correlationvalue is not smaller than the threshold. The threshold is a lower limitof the correlation value, and may be advantageously used in extractingonly a candidate with high search reliability. Note that the evaluationvalue in the sorting step and the correlation value in the extractionstep may be indexes different from each other. For example, a ratio ofedge pixels may be used as the evaluation value. The ratio of edgepixels is a robust index with respect to variation in scale, and henceit may be advantageously used in positioning the inspection target 8with its scale apt to vary. The correlation value may be an indexconcerning a similarity of a contour (edge) included in the standardregion, and the evaluation value may be an index concerning the numberof contour pixels (edge pixels) included in the sorting region.

According to S24, there are provided a decision step of deciding aposition and a posture of an inspection region of a visual inspectiontool that performs visual inspection on the inspection target 8 in linewith a position and a posture of the candidate for the standard pattern,and a performing step of performing the visual inspection by use of thevisual inspection tool in the inspection region disposed with respect tothe inspection target image. According to the present embodiment, sincethe accuracy in searching the position and posture of the candidate forthe standard pattern in the inspection image improves, the accuracy indeciding the position and posture of the inspection region of the visualinspection tool also improves. Eventually, the accuracy in visualinspection also improves.

As described concerning the sorting enabling section 54 and the checkbox 82 for switching the enablement and disablement of sorting, theremay be further provided a selection step of selecting whether or not toperform the sorting step. The selection step may be part of the settingstep (S4 of FIG. 3) for the position/posture correction. When performingthe sorting step is selected in this selection step, the output stepoutputs the candidate for the standard pattern, sorted in the sortingstep. On the other hand, when not performing the sorting step isselected in the selection step, the output step outputs the candidatefor the standard pattern, extracted in the extraction step. It ispossible to sufficiently extract the candidate for the standard patterneven in the standard region alone depending on the shape of the standardpattern itself and the shape of the periphery of the standard pattern.In such a case, it may be desirable to omit the sorting step and reducethe processing time taken throughout the positioning method.

As described using FIG. 13 and the like, the sorting region is disposedwith respect to each of the plurality of candidates for the standardpattern in the sorting step so as to hold a relative positional relationwith respect to the standard region set in the setting step. Forexample, when the position and posture of the standard region change,the sorting region is decided according to the position and posture ofthe standard region. Further, when the change in scale is to beconsidered, the scale of the sorting region is adjusted so as to holdthe relative positional relation with respect to the standard region.

As described using FIG. 6A, the standard region may be set so as tosurround the alignment mark provided in the inspection target 8, and thesorting region may be set around the alignment mark. On the inspectiontarget 8, the alignment mark may be marked or printed apart from theperipheral shape. In this case, a flat edgeless region extends betweenthe alignment mark and the peripheral shape. On the other hand, the edgemay be present around the shape similar to the alignment mark. Thus, aregion being around the alignment mark and not having an edge or thelike may be taken as the sorting region, to make the alignment markeasily distinguishable from the similar shape. That is, the sortingregion may be set around the standard pattern, which is helpful fordistinguishing between the standard pattern and the similar shape.

As described using FIG. 6C, the sorting region may be set with respectto the feature for identifying the front surface and the rear surface ofthe inspection target 8. This allows accurate identification of thefront surface and the rear surface of the inspection target 8. Further,in addition to the sorting region for distinguishing the front and therear, a sorting region for ensuring a grasping region for the robot handmay be simultaneously set. These two sorting regions may each be definedby a plurality of (e.g., four) detached regions.

Note that in principle, the standard region may be defined not bydetached regions, and the sorting region may be defined by a pluralityof detached regions. For example, the reason the sorting region forgrasping by the robot hand is made up of detached regions is because therobot hand has at least two fingers and grasps the inspection target 8so as to pinch it with those fingers. That is, since the plurality offingers come into contact with different regions of the surface of theinspection target 8, a plurality of separate contact portions are set asthe sorting region.

As described using FIG. 6B, the sorting regions may be disposed aroundthe standard region in order to detect the presence or absence of anobstacle that becomes an obstacle when the inspection target 8 isgrasped by the robot handle. When an obstacle is present around theinspection target 8, the robot-picking may fail. Thus, positioning therobot hand with respect to the inspection target 8 with no obstaclepresent therearound may lead to improvement in robot-picking successratio.

The foregoing positioning processing and applications may be stored ascontrol programs in the program memory 24. In this case, the CPU 22 andthe image processing section 30 each function as a sort of computer, andruns the control program stored in the program memory 24, to performeach step. The image processing section 30 may have a second CPU forrunning the control program, or the image processing section 30 may beintegrated into the CPU 22. Note that the control program may berecorded in a computer readable recording medium and provided.

The foregoing visual inspection apparatus 1 and controller 2 are anexample of the positioning apparatus for positioning a standard patternwith respect to an inspection image. The monitor 10 functions as adisplay unit for displaying a standard image of a product to be astandard for the inspection target 8. The standard region settingsection 52 and the sorting region setting section 53 function as asetting unit for setting a standard region so as to surround thestandard pattern in the standard image, and setting a sorting region,which is a region for sorting a plurality of candidates similar to thestandard pattern in the inspection image. The candidate extractingsection 64 functions as an extraction unit for searching the standardpattern from the inspection image to extract a plurality of candidatessimilar to the standard pattern. The candidate sorting section 65functions as a sorting unit for disposing the sorting region withrespect to the plurality of candidates for the extracted standardpattern, to sort a candidate for the standard pattern based on anevaluation value of the sorting region disposed with respect to each ofthe plurality of candidates for the standard pattern. The output section66 functions as an output unit for outputting a candidate for the sortedstandard pattern.

The evaluation value has been exemplified by the number of edge pixelsand the ratio of edge pixels, but the number of edge pixels by directionmay be employed. Further, dispersion of edges in the sorting region 41,a scratch value detected by a scratch inspection unit, or an edge widthmeasured by an edge width measurement unit may be employed.

Further, a feature value calculated from the sorting region in theinspection image may be compared with a feature value calculated fromthe sorting region for each candidate in the standard image, to narrowdown a plurality of candidates into candidates having similar features.As the feature value, there can be employed a difference absolute sum, amutual information amount, a normalized correlation value, gradation ora distance of a histogram of an edge strength or an edge angle, and thelike.

Note that the user may instruct to display all candidates extracted inthe extraction step. In this case, the output section may makeintensified display for all the candidates for the standard pattern 31found in the inspection image so as to show that those are thecandidates. Further, the output section may distinguish the candidatessorted out of the candidates for the standard pattern 31 found in theinspection image by another intensified display. Moreover, the outputsection may apply still another intensified display to the bestcandidate of the sorted candidates. For example, a yellow frame may bedisplayed for each of the candidates for the standard pattern 31 foundin the inspection image, a red frame may be displayed for each of thesorted candidates, and a green frame may be displayed for the bestcandidate.

The application has been exemplified by detecting the alignment mark,performing the robot-picking, distinguishing the front and the rear, andthe like. Naturally, however, the present embodiment can also beemployed in other applications. There may be employed an applicationwhere a sorting region is provided in a feature section thatdistinguishes one product type from another, to identify a product type.There may be employed an application where a sorting region is setaround one workpiece to detect an overlap of the one workpiece withanother workpiece. There may be employed an application where a sortingregion is set around a workpiece to detect an overlap of the workpiecewith an obstacle. There may be employed an application where a workpiecewithout a scratch is sorted in a sorting region, or an application wherea workpiece with a scratch is removed from a sorting region. There maybe employed an application of sorting a workpiece where a tolerance of asize of a portion set in a sorting region is in an allowable range.There may be employed an application where adhesion of a foreign matteris detected in a sorting region, to remove a workpiece with the foreignmatter adhering thereto or to sort a workpiece without the foreignmatter.

In FIG. 13, a circumscribed rectangle has been defined for each sortingregion 41, but a single circumscribed rectangle may be defined for aplurality of sorting regions 41. That is, the smallest rectanglesurrounding a plurality of sorting regions 41 may be defined. This mayreduce a calculation amount.

What is claimed is:
 1. A positioning method for searching a previouslyregistered standard pattern out of an inspection target image obtainedby capturing an image of an inspection target, to position the standardpattern with respect to the inspection target image, so as to conduct apass/fail determination of a product or sorting of a product, whereinthe inspection target is the product manufactured in a factory themethod comprising: a setting step of displaying a standard image of aproduct to be a standard for the inspection target and setting a firstregion in the standard image so as to surround the standard pattern inthe standard image and a second region in the standard image, the secondregion being a region for sorting a plurality of candidates similar tothe standard pattern in the inspection target image; a storing step ofstoring a relative positional relation of the second region with respectto the first region set in the setting step; an extraction step ofsearching the standard pattern from the inspection target image,extracting the plurality of candidates similar to the standard patternfrom the inspection target image, and deciding the position of the firstregion corresponding to each of the plurality of candidates; a sortingstep of disposing the second region with respect to each of theplurality of candidates extracted in the extraction step so as to havesaid relative positional relation with respect to the position of thefirst region decided in the extraction step, and sorting the pluralityof candidates extracted in the extraction step based on an evaluationvalue of the second region disposed with respect to each of theplurality of candidates for the standard pattern; and an output step ofoutputting the desired candidate for the standard pattern based on asorting result of the sorting step.
 2. The positioning method accordingto claim 1, wherein the sorting step includes a determination step ofdetermining whether the evaluation value of the second region disposedwith respect to each of the plurality of candidates for the standardpattern satisfies a predetermined condition.
 3. The positioning methodaccording to claim 2, wherein the determination step determines whetherthe evaluation value of the second region concerning the plurality ofcandidates for the standard pattern, extracted from the inspectiontarget image, exceeds a threshold.
 4. The positioning method accordingto claim 1, wherein the extraction step includes a step of extracting aplurality of candidates similar to the standard pattern based on acorrelation value of the standard pattern and the inspection targetimage, and the evaluation value in the sorting step and the correlationvalue in the extraction step are indexes different from each other. 5.The positioning method according to claim 4, wherein the correlationvalue is an index concerning a similarity of a contour included in thefirst region, and the evaluation value is an index concerning the numberof contour pixels included in the second region.
 6. The positioningmethod according to claim 1, further comprising: a decision step ofdeciding a position and a posture of an inspection region of a visualinspection tool that performs visual inspection on the inspection targetin line with a position and a posture of the candidate for the standardpattern; and a performing step of performing the visual inspection byuse of the visual inspection tool in the inspection region disposed withrespect to the inspection target image.
 7. The positioning methodaccording to claim 1, further comprising a selection step of selectingwhether or not to perform the sorting step, wherein, when performing thesorting step is selected in the selection step, the output step outputsthe candidate for the standard pattern, sorted in the sorting step, andwhen not performing the sorting step is selected in the selection step,the output step outputs the candidate for the standard pattern,extracted in the extraction step.
 8. The positioning method according toclaim 1, wherein the first region is set so as to surround an alignmentmark provided on the inspection target, and the second region is setaround the alignment mark.
 9. The positioning method according to claim1, wherein the second region is set with respect to a feature foridentifying a front surface and a rear surface of the inspection target.10. The positioning method according to claim 1, wherein the secondregion is disposed around the first region in order to detect presenceor absence of an obstacle that becomes an obstacle when the inspectiontarget is grasped by a robot handle.
 11. The positioning methodaccording to claim 1, wherein the second region is made up of aplurality of detached regions.
 12. A non-transitory computer readablerecording medium, comprising a computer program which executes thepositioning method according to claim
 1. 13. A positioning apparatus forsearching a previously registered standard pattern out of an inspectiontarget image obtained by capturing an image of an inspection target, toposition the standard pattern with respect to the inspection targetimage, so as to conduct a pass/fail determination of a product orsorting of a product, wherein the inspection target is the productmanufactured in a factory the apparatus comprising: a setting unit fordisplaying a standard image of a product to be a standard for theinspection target and setting a first region in the standard image so asto surround the standard pattern in the standard image and a secondregion in the standard image, the second region being a region forsorting a plurality of candidates similar to the standard pattern in theinspection target image; a storing unit for storing a relativepositional relation of the second region with respect to the firstregion set by the setting unit; an extraction unit for searching thestandard pattern from the inspection target image, extracting theplurality of candidates similar to the standard pattern from theinspection target image, and deciding the position of the first regioncorresponding to each of the plurality of candidates; a sorting unit fordisposing the second region with respect to each of the plurality ofcandidates, extracted by the extraction unit so as to have said relativepositional relation with respect to the position of the first regionprovided by the extraction unit, and sorting the plurality of candidatesextracted by the extraction unit based on an evaluation value of thesecond region disposed with respect to each of the plurality ofcandidates for the standard pattern; and an output unit for outputtingthe desired candidate for the standard pattern based on a sorting resultof the sorting unit.